Part 13.1 - Machine Learning Python Programming Concepts

Agenda

• Introduction of Machine Learning
• Machine Learning current implementation and implementation
• Machine Learning Concepts – Hypothesis function, Cost function, Gradient Descent, Learning Rate
• Machine Learning Process / Workflow
• Machine Learning Types – Supervised and Unsupervised
• Machine Learning Algorithms: SVM, KNN, Decision Tree, Random Forest, Logistic Regression, Linear Regression, K-means Clustering
• Artificial Neural Network (ANN)
• Activation Functions
• Loss function
• Optimizer
• Deep Neural Network (DNN)
• Convolutional Neural Network (CNN)
• Recurrent Neural Network (RNN)
• Transfer Learning

Honey, do you know about Machine Learning?

Many peopole around us expects us to know about Machine Learning. Why?

• Programming
• Statistics / Modeling
• Working with data all the times

Big Picture of Data Science, AI, Machine Learrning, Deep Learning

## Left aligned image

• Data Science : a field of pertaining to data systems and processes, aimed at maintaining data sets and deriving meaning out of data.
• AI (Artificial Intelligence) : a filed of enabling machines to execute reasoning by replicating human intelligence.
• Machine Learning : a subset of AI that Machine/system can automatically learn and improve from experience.
• Deep Learning : a subset of Machine Learning that system uses artifical neural network to learn.

Current AI Implementation

Voice Regnition

Recommendation - Amazon, Netflix, Spotify

Chatbot (Online Customer Service)

AI Assistant (AI System for accomplishing Real-World Tasks over the phone)

Amazon Go (Casher less Grocery Store)

AlphaGO – Beat “Go” World Champion

Why is AI(ML) so popular now?

• Innovative
  • Can solve many complex business problems that were not resolved before
• New business development
• Cost effective - 24 hours / 7 days

• Automate a lot of works

  • Can replace or enhance human labors
  • “Pretty much anything that a normal person can do in <1 sec, we can now automate with AI” Andrew Ng

• Accurate

  • Better than humans in some fields / applications

AI Market Expected Growth

The global AI market is expected to grow from 40 billion in 2020 to 51 billion in 2021 at a compound annual growth rate (CAGR) of 28.4%. The market is expected to reach 171 billion in 2025 at a CAGR of 35%.

Machine Learning Introduction

An application of artificial intelligence (AI) that provides systems the ability to automatically learn and improve from experience without being explicitly programmed.

What is it?

How does human learn?

• Through Experiences

How does Machine learn?

• Data
• Algorithm (Hypothesis Function)

Machine Learning Functions

Hypothesis function

• ML Algorithm/Model that we will train with data
• Linear Regression, Logistic Regression, Support Vector Machine, Decision Tree, Artificial Neural Network
• hθ(x) = θ0 x0 + θ1 x1 + θ2 x2 + θ3 x3 +,, + θn* xn (e.g., Y = 2x + 30)
• Parameters (ultimate goals of ML) : θ0, θ1, θ2 , θ3,,,θn

Cost function (loss function)

• To measure how well hypothesis function fits into data.
• Measured Difference between actual data point and hypothesized data point. (e.g., Y – hθ(x))
• J(θ0, θ1, θ2,,θn) = (1/2n)*sum[(Y - H)^2] for linear model ( Square Error Cost function)

Gradient Descent (optimizer)

• Engine that minimizes cost function
• Repeat
  • θ0 := θ0 – alpha d/dθ0 J(θ0, θ1, θ2,,θn)
  • θ1 := θ1 – alpha d/dθ1 J(θ0, θ1, θ2,,θn)
• Learning rate (alpha) – size of learning step in gradient descent

How Machine Learning works

Hypothesis function vs cost function vs gradient descent

image-2.png

Teaching Young Childrens about Cat Pictures

1. Show first sets of pictures
2. Tell kids that some are “cat” pictures and some are NOT.
3. Explain why some are NOT “cat” pictures
4. Show next sets of pictures
5. Repeat 2-3
6. Keep showing CAT pictures using step 4 and 5 until kids understand “cat” pictures.

Machine Learning Process

1. Select Data and hypothesis function
2. Start with random parameters of hypothesis function
3. Process first sets of data through hypothesis function
4. Find predicted values from hypothesis function
5. Calculate cost function ( actual – predicted)
6. Minimize cost function using Gradient Descent
7. Update parameters of hypothesis function based on cost function minimization
8. Repeat steps ( 3 and 7) with next sets of data until the cost function become 0 or at the lowest point.

1. Select Data and Hypothesis Function

Data :

• Y = Survival Days in months
• X = number of drug exposures

Hypothesis function:

• Linear Regression (hθ(x) = θx)

image-2.png

2. Start with random parameters of hypothesis function

hθ(x) = 4x at θ = 4

3 & 4 Process first set of data in hypothesis function and predict values from hypothesis function

hθ(x) = 4x at θ = 4

x = [1,2,3]

• at x = 1, hθ(1) = 4
• at x = 2, hθ(2) = 8
• at x = 3, hθ(3) = 12

5. Calculate cost function : difference between actual and predicted

Cost function : J(θ)=(1/2m)*sum[(Y - h)^2]

J(4) =(1/6)*sum((2-4)^2 + (4-8)^2 + (6-12)^2) = 56/6 = 9.333

image-2.png

Cost Function in different Parameter

• J(0) =56/6 = 9.333
• J(1) =14/6 = 2.333
• J(2) =0/6 = 0
• J(3) =14/6 = 2.333
• J(4) =56/6 = 9.333

Optimum θ is 2 : Best fitted model is H = 2X.

image-2.png

Gradient Descent

• Definition – Engine that minimize cost function.
• The process of getting to the lowest error value (optimum cost function)
• It is like walking down in the mountain to find the gold located in valley.
  

6. Minimize cost function using Gradient Descent

• Take the derivative of cost function since it will give the direction to move forward : d/dθ J(θ)
• Make steps down the cost function : θ := θ – d/dθ J(θ)
• The size of steps is determined by learning rate : θ := θ – alpha* d/dθ J(θ)

image-2.png

Learning Rate ( alpha)

At θ:= θ – alpha* d/dθJ(θ) at θ =4 assuming d/dθJ(4) = 2.

If the learning rate (alpha) = 0.1,
θ:= 4 - 0.1 2 = 3.8
If the learning rate (alpha) = 0.5
θ:= 4 – 0.5 2 = 3

7. Update parameters of hypothesis function based on cost function

• Initial Hypothesis function - hθ(x) = 4x

• Cost function - J(θ) = (1/2m)sum[(Y - h)^2] = 1/(23)* sum(((2-4)^2 + (4-8)^2 + (6-12))^2)) = 56/6=9.333

• Gradient Descent – θ := θ – alpha d/dθJ(4) = 4 – 0.12 = 3.8

• Modified Hypothesis function - hθ(x) = 3.8x

8.Repeat steps ( 3 and 7) with next sets of data

At second iteration,

• Initial Hypothesis function - hθ(x) = 3.8x

• Cost function - J(θ) = 1/(23) sum(((2-3.8)^2 + (4-7.6)^2 + (6-11.4)^2)) = 45.36/6 = 7.56

• Gradient Descent – θ := θ – alpha d/dθ J(3.8) = 3.8 – 0.12 = 3.6

• Final Hypothesis function - hθ(x) = 3.6x

And the iteration will go on until the cost function gest to 0 like J(θ)=0 or it no longer decreases or no more data.

The cost function decreases and slope (θ) decreases as well to the optimum point.

Machine Learning Interview

Interviewer: What’s your biggest strength ?
Me: I’m an expert in Machine Learning.
Interviewer: What’s 9 + 10 ?
Me: It is 3.
Interviewer: Not even close. It’s 19.
Me: It’s 16.
Interviewer: Wrong. It's still 19.
Me: It’s 18.
Interviewer: No, it’s 19… Arghhh
Me: It’s 19.
Interviewer: You are hired.